Natural Language Processing Overview

Principles

Definition

Natural language processing is an interdisciplinary field at the intersection of computer science, artificial intelligence, and linguistics dedicated to enabling computers to understand, interpret, and generate human (natural) languages. It investigates the computational theories, methodologies, and algorithms necessary to process, analyze, and manipulate human language data in textual and spoken forms, aiming to develop systems capable of performing language-related tasks effectively and bridging the gap between human communication and machine comprehension.

Ontological type

Algorithmic Approaches

Core Tasks

Data and Resources

Key developments

Key Figures

Symbolic Rule-Based NLP era

During the Symbolic Rule-Based NLP era (1963–1988), researchers built hand-crafted grammars, augmented transition networks, and knowledge-rich semantic representations to map syntax to meaning. Terry Winograd's SHRDLU (1968) demonstrated interactive, goal-directed dialogue and robust parsing within a blocks-world domain, epitomizing rule-based interpretation. Roger Schank and Robert Abelson's Conceptual Dependency theory and Script theory encoded deep semantic relations and event structures to support understanding and reasoning. Marvin Minsky's frame-based representations provided a foundational knowledge architecture for organizing world knowledge in symbolic NLP systems.

Statistical Data-Driven NLP era

Michael I. Jordan [1] is a leading figure associated with the Institute of Cognitive and Brain Sciences [3] and the Massachusetts Institute of Technology [4] during the statistical data-driven NLP era. His work on Latent dirichlet allocation [7] introduced a probabilistic topic model that enables unsupervised discovery of latent structure in large text corpora, a breakthrough that underpinned many data-driven NLP methods in this era. David M. Blei [2] is affiliated with the University of California, Berkeley [5] and Princeton University [6] during this era. As co-author of Latent dirichlet allocation [7], he helped establish topic modeling as a central tool for discovering latent structure in text, a cornerstone of statistical NLP in this period.

Neural Pretraining and Transformers era

Christopher D. Manning[1] is a leading AI/NLP scholar associated with Harvard University[3] and Stanford University[4] during the Neural Pretraining and Transformers era. His key contributions include Glove: Global Vectors for Word Representation[6] and Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank[7], which advanced distributed word representations and hierarchical composition and underpinned later transfer and pretraining approaches in this era. Richard Socher[2] is a prominent NLP researcher affiliated with Stanford University[4] and the University of Colorado Boulder[5] during the Neural Pretraining and Transformers era. His notable contributions include Glove: Global Vectors for Word Representation[6] and Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank[7], illustrating how robust word representations and recursive composition informed early neural architectures and influenced transformer-based pretraining strategies.